Benchmarking reports

from sklearn_benchmarks.report import Reporting
import pandas as pd
pd.set_option('display.max_colwidth', None)
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

reporting = Reporting(config_file_path="config.yml")
reporting.run()

Time report

	hour	min	sec
algo
KNeighborsClassifier	0.0	13.0	19.660085
daal4py_KNeighborsClassifier	0.0	5.0	10.622561
KNeighborsClassifier_kd_tree	0.0	5.0	52.867885
daal4py_KNeighborsClassifier_kd_tree	0.0	1.0	45.414334
KMeans_tall	0.0	1.0	45.612841
daal4py_KMeans_tall	0.0	1.0	18.235142
KMeans_short	0.0	0.0	23.888986
daal4py_KMeans_short	0.0	0.0	13.454837
LogisticRegression	0.0	1.0	7.549737
daal4py_LogisticRegression	0.0	0.0	59.564112
Ridge	0.0	0.0	27.080876
daal4py_Ridge	0.0	0.0	8.942399
total	0.0	32.0	32.975178

KNeighborsClassifier vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	brute

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	n_jobs	n_neighbors	throughput	latency	accuracy_score_sklearn	accuracy_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KNeighborsClassifier	fit	0.130	0.003	1000000	1000000	100	-1	1	6.174	NaN	0.985	0.983	0.495	0.013	0.262	0.010	See
1	KNeighborsClassifier	predict	31.838	0.000	1000000	1000	100	-1	1	0.000	0.032	0.985	0.983	3.773	0.037	8.438	0.083	See
2	KNeighborsClassifier	predict	0.189	0.011	1000000	1	100	-1	1	0.004	0.000	0.985	0.983	0.103	0.002	1.834	0.116	See
3	KNeighborsClassifier	fit	0.126	0.005	1000000	1000000	100	-1	5	6.359	NaN	0.985	0.983	0.504	0.015	0.249	0.012	See
4	KNeighborsClassifier	predict	36.199	0.000	1000000	1000	100	-1	5	0.000	0.036	0.985	0.983	3.786	0.019	9.561	0.049	See
5	KNeighborsClassifier	predict	0.209	0.010	1000000	1	100	-1	5	0.004	0.000	0.985	0.983	0.108	0.004	1.932	0.117	See
6	KNeighborsClassifier	fit	0.131	0.003	1000000	1000000	100	-1	100	6.129	NaN	0.985	0.983	0.503	0.010	0.260	0.009	See
7	KNeighborsClassifier	predict	36.197	0.000	1000000	1000	100	-1	100	0.000	0.036	0.985	0.983	3.845	0.023	9.415	0.056	See
8	KNeighborsClassifier	predict	0.211	0.017	1000000	1	100	-1	100	0.004	0.000	0.985	0.983	0.102	0.002	2.077	0.170	See
9	KNeighborsClassifier	fit	0.141	0.004	1000000	1000000	100	1	1	5.693	NaN	0.985	0.983	0.512	0.010	0.274	0.009	See
10	KNeighborsClassifier	predict	15.448	0.044	1000000	1000	100	1	1	0.000	0.015	0.985	0.983	3.786	0.026	4.080	0.031	See
11	KNeighborsClassifier	predict	0.209	0.009	1000000	1	100	1	1	0.004	0.000	0.985	0.983	0.106	0.003	1.963	0.096	See
12	KNeighborsClassifier	fit	0.130	0.005	1000000	1000000	100	1	5	6.143	NaN	0.985	0.983	0.511	0.014	0.255	0.012	See
13	KNeighborsClassifier	predict	22.061	0.034	1000000	1000	100	1	5	0.000	0.022	0.985	0.983	3.774	0.015	5.845	0.026	See
14	KNeighborsClassifier	predict	0.224	0.009	1000000	1	100	1	5	0.004	0.000	0.985	0.983	0.106	0.003	2.113	0.100	See
15	KNeighborsClassifier	fit	0.125	0.004	1000000	1000000	100	1	100	6.381	NaN	0.985	0.983	0.501	0.014	0.250	0.011	See
16	KNeighborsClassifier	predict	22.182	0.037	1000000	1000	100	1	100	0.000	0.022	0.985	0.983	3.823	0.021	5.803	0.033	See
17	KNeighborsClassifier	predict	0.220	0.008	1000000	1	100	1	100	0.004	0.000	0.985	0.983	0.106	0.002	2.064	0.086	See
18	KNeighborsClassifier	fit	0.055	0.001	1000000	1000000	2	-1	1	0.292	NaN	0.985	0.983	0.095	0.003	0.579	0.026	See
19	KNeighborsClassifier	predict	25.081	0.190	1000000	1000	2	-1	1	0.000	0.025	0.985	0.983	0.794	0.022	31.580	0.923	See
20	KNeighborsClassifier	predict	0.021	0.001	1000000	1	2	-1	1	0.001	0.000	0.985	0.983	0.004	0.001	4.801	0.682	See
21	KNeighborsClassifier	fit	0.053	0.001	1000000	1000000	2	-1	5	0.300	NaN	0.985	0.983	0.094	0.004	0.565	0.028	See
22	KNeighborsClassifier	predict	32.716	0.000	1000000	1000	2	-1	5	0.000	0.033	0.985	0.983	0.791	0.014	41.357	0.707	See
23	KNeighborsClassifier	predict	0.032	0.002	1000000	1	2	-1	5	0.001	0.000	0.985	0.983	0.004	0.000	7.062	0.732	See
24	KNeighborsClassifier	fit	0.054	0.002	1000000	1000000	2	-1	100	0.299	NaN	0.985	0.983	0.096	0.002	0.556	0.024	See
25	KNeighborsClassifier	predict	32.336	0.000	1000000	1000	2	-1	100	0.000	0.032	0.985	0.983	0.870	0.023	37.181	0.970	See
26	KNeighborsClassifier	predict	0.032	0.002	1000000	1	2	-1	100	0.001	0.000	0.985	0.983	0.004	0.001	7.162	1.088	See
27	KNeighborsClassifier	fit	0.053	0.001	1000000	1000000	2	1	1	0.301	NaN	0.985	0.983	0.098	0.005	0.540	0.030	See
28	KNeighborsClassifier	predict	10.519	0.210	1000000	1000	2	1	1	0.000	0.011	0.985	0.983	0.792	0.015	13.280	0.368	See
29	KNeighborsClassifier	predict	0.015	0.002	1000000	1	2	1	1	0.001	0.000	0.985	0.983	0.004	0.001	3.542	0.640	See
30	KNeighborsClassifier	fit	0.054	0.001	1000000	1000000	2	1	5	0.298	NaN	0.985	0.983	0.094	0.002	0.574	0.018	See
31	KNeighborsClassifier	predict	18.395	0.047	1000000	1000	2	1	5	0.000	0.018	0.985	0.983	0.787	0.011	23.381	0.342	See
32	KNeighborsClassifier	predict	0.027	0.002	1000000	1	2	1	5	0.001	0.000	0.985	0.983	0.004	0.000	6.560	0.887	See
33	KNeighborsClassifier	fit	0.053	0.001	1000000	1000000	2	1	100	0.300	NaN	0.985	0.983	0.094	0.004	0.565	0.029	See
34	KNeighborsClassifier	predict	18.370	0.070	1000000	1000	2	1	100	0.000	0.018	0.985	0.983	0.851	0.017	21.589	0.431	See
35	KNeighborsClassifier	predict	0.026	0.002	1000000	1	2	1	100	0.001	0.000	0.985	0.983	0.006	0.001	4.317	1.012	See

KNeighborsClassifier_kd_tree vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	kd_tree

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	n_jobs	n_neighbors	throughput	latency	accuracy_score_sklearn	accuracy_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KNeighborsClassifier_kd_tree	fit	2.694	0.041	1000000	1000000	10	-1	1	0.030	NaN	0.99	0.984	0.696	0.019	3.872	0.120	See
1	KNeighborsClassifier_kd_tree	predict	0.450	0.014	1000000	1000	10	-1	1	0.000	0.000	0.99	0.984	0.122	0.005	3.681	0.183	See
2	KNeighborsClassifier_kd_tree	predict	0.003	0.000	1000000	1	10	-1	1	0.026	0.000	0.99	0.984	0.000	0.000	12.388	5.145	See
3	KNeighborsClassifier_kd_tree	fit	2.699	0.034	1000000	1000000	10	-1	5	0.030	NaN	0.99	0.984	0.757	0.012	3.564	0.073	See
4	KNeighborsClassifier_kd_tree	predict	0.874	0.017	1000000	1000	10	-1	5	0.000	0.001	0.99	0.984	0.213	0.004	4.098	0.109	See
5	KNeighborsClassifier_kd_tree	predict	0.004	0.001	1000000	1	10	-1	5	0.023	0.000	0.99	0.984	0.000	0.000	7.321	2.937	See
6	KNeighborsClassifier_kd_tree	fit	2.714	0.036	1000000	1000000	10	-1	100	0.029	NaN	0.99	0.984	0.690	0.019	3.931	0.121	See
7	KNeighborsClassifier_kd_tree	predict	2.872	0.034	1000000	1000	10	-1	100	0.000	0.003	0.99	0.984	0.688	0.014	4.177	0.099	See
8	KNeighborsClassifier_kd_tree	predict	0.005	0.001	1000000	1	10	-1	100	0.015	0.000	0.99	0.984	0.001	0.000	6.002	2.892	See
9	KNeighborsClassifier_kd_tree	fit	2.726	0.063	1000000	1000000	10	1	1	0.029	NaN	0.99	0.984	0.738	0.017	3.693	0.122	See
10	KNeighborsClassifier_kd_tree	predict	0.753	0.020	1000000	1000	10	1	1	0.000	0.001	0.99	0.984	0.122	0.006	6.148	0.344	See
11	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	10	1	1	0.065	0.000	0.99	0.984	0.000	0.000	5.337	2.771	See
12	KNeighborsClassifier_kd_tree	fit	2.690	0.038	1000000	1000000	10	1	5	0.030	NaN	0.99	0.984	0.687	0.013	3.914	0.093	See
13	KNeighborsClassifier_kd_tree	predict	1.458	0.027	1000000	1000	10	1	5	0.000	0.001	0.99	0.984	0.228	0.004	6.410	0.163	See
14	KNeighborsClassifier_kd_tree	predict	0.002	0.000	1000000	1	10	1	5	0.047	0.000	0.99	0.984	0.000	0.000	3.907	1.846	See
15	KNeighborsClassifier_kd_tree	fit	2.735	0.058	1000000	1000000	10	1	100	0.029	NaN	0.99	0.984	0.733	0.015	3.730	0.110	See
16	KNeighborsClassifier_kd_tree	predict	4.798	0.074	1000000	1000	10	1	100	0.000	0.005	0.99	0.984	0.697	0.023	6.880	0.254	See
17	KNeighborsClassifier_kd_tree	predict	0.004	0.001	1000000	1	10	1	100	0.022	0.000	0.99	0.984	0.001	0.000	4.121	2.176	See
18	KNeighborsClassifier_kd_tree	fit	0.741	0.023	1000000	1000000	2	-1	1	0.022	NaN	0.99	0.984	0.475	0.011	1.561	0.061	See
19	KNeighborsClassifier_kd_tree	predict	0.032	0.002	1000000	1000	2	-1	1	0.001	0.000	0.99	0.984	0.001	0.000	40.871	11.693	See
20	KNeighborsClassifier_kd_tree	predict	0.009	0.020	1000000	1	2	-1	1	0.002	0.000	0.99	0.984	0.000	0.000	59.130	131.071	See
21	KNeighborsClassifier_kd_tree	fit	0.729	0.015	1000000	1000000	2	-1	5	0.022	NaN	0.99	0.984	0.476	0.009	1.530	0.044	See
22	KNeighborsClassifier_kd_tree	predict	0.037	0.010	1000000	1000	2	-1	5	0.000	0.000	0.99	0.984	0.001	0.000	32.002	13.976	See
23	KNeighborsClassifier_kd_tree	predict	0.003	0.001	1000000	1	2	-1	5	0.006	0.000	0.99	0.984	0.000	0.000	18.176	10.967	See
24	KNeighborsClassifier_kd_tree	fit	0.727	0.011	1000000	1000000	2	-1	100	0.022	NaN	0.99	0.984	0.477	0.017	1.524	0.058	See
25	KNeighborsClassifier_kd_tree	predict	0.050	0.003	1000000	1000	2	-1	100	0.000	0.000	0.99	0.984	0.007	0.001	7.004	0.809	See
26	KNeighborsClassifier_kd_tree	predict	0.003	0.000	1000000	1	2	-1	100	0.006	0.000	0.99	0.984	0.000	0.000	11.521	7.471	See
27	KNeighborsClassifier_kd_tree	fit	0.726	0.014	1000000	1000000	2	1	1	0.022	NaN	0.99	0.984	0.476	0.016	1.524	0.059	See
28	KNeighborsClassifier_kd_tree	predict	0.029	0.002	1000000	1000	2	1	1	0.001	0.000	0.99	0.984	0.001	0.000	35.723	11.396	See
29	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	1	0.020	0.000	0.99	0.984	0.000	0.000	4.829	2.801	See
30	KNeighborsClassifier_kd_tree	fit	0.725	0.014	1000000	1000000	2	1	5	0.022	NaN	0.99	0.984	0.478	0.010	1.516	0.043	See
31	KNeighborsClassifier_kd_tree	predict	0.032	0.001	1000000	1000	2	1	5	0.001	0.000	0.99	0.984	0.001	0.000	25.985	8.684	See
32	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	5	0.019	0.000	0.99	0.984	0.000	0.000	4.886	2.572	See
33	KNeighborsClassifier_kd_tree	fit	0.717	0.013	1000000	1000000	2	1	100	0.022	NaN	0.99	0.984	0.476	0.010	1.507	0.042	See
34	KNeighborsClassifier_kd_tree	predict	0.056	0.004	1000000	1000	2	1	100	0.000	0.000	0.99	0.984	0.007	0.001	7.935	0.958	See
35	KNeighborsClassifier_kd_tree	predict	0.001	0.000	1000000	1	2	1	100	0.020	0.000	0.99	0.984	0.000	0.000	4.570	2.371	See

KMeans_tall vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	full
n_clusters	3
max_iter	30
n_init	1
tol	0.0

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	init	n_iter_sklearn	throughput	latency	adjusted_rand_score_sklearn	n_iter_daal4py	adjusted_rand_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KMeans_tall	fit	0.588	0.019	1000000	1000000	2	k-means++	30	0.816	NaN	0.002	30	0.003	0.304	0.010	1.932	0.091	See
1	KMeans_tall	predict	0.002	0.000	1000000	1000	2	k-means++	30	0.010	0.0	0.002	30	0.003	0.000	0.000	7.276	3.651	See
2	KMeans_tall	predict	0.001	0.000	1000000	1	2	k-means++	30	0.011	0.0	0.002	30	0.003	0.000	0.000	8.823	4.298	See
3	KMeans_tall	fit	0.502	0.013	1000000	1000000	2	random	30	0.957	NaN	0.002	30	0.003	0.260	0.013	1.926	0.106	See
4	KMeans_tall	predict	0.002	0.000	1000000	1000	2	random	30	0.010	0.0	0.002	30	0.003	0.000	0.000	7.837	3.651	See
5	KMeans_tall	predict	0.002	0.000	1000000	1	2	random	30	0.011	0.0	0.002	30	0.003	0.000	0.000	7.707	3.547	See
6	KMeans_tall	fit	7.131	0.129	1000000	1000000	100	k-means++	30	3.365	NaN	0.002	30	0.003	3.947	0.066	1.807	0.045	See
7	KMeans_tall	predict	0.002	0.000	1000000	1000	100	k-means++	30	0.466	0.0	0.002	30	0.003	0.000	0.000	5.921	2.384	See
8	KMeans_tall	predict	0.002	0.000	1000000	1	100	k-means++	30	0.498	0.0	0.002	30	0.003	0.000	0.000	9.217	4.943	See
9	KMeans_tall	fit	6.617	0.049	1000000	1000000	100	random	30	3.627	NaN	0.002	30	0.003	3.788	0.056	1.747	0.029	See
10	KMeans_tall	predict	0.002	0.000	1000000	1000	100	random	30	0.424	0.0	0.002	30	0.003	0.000	0.000	6.581	2.810	See
11	KMeans_tall	predict	0.002	0.001	1000000	1	100	random	30	0.471	0.0	0.002	30	0.003	0.000	0.000	9.667	5.285	See

KMeans_short vs daal4py

All estimators share the following hyperparameters:

	value
algorithm	full
n_clusters	300
max_iter	30
n_init	1
tol	0.0

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	init	n_iter_sklearn	throughput	latency	adjusted_rand_score_sklearn	n_iter_daal4py	adjusted_rand_score_daal4py	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	KMeans_short	fit	0.314	0.014	10000	10000	2	k-means++	30	0.015	NaN	0.005	30	0.003	0.147	0.004	2.132	0.110	See
1	KMeans_short	predict	0.002	0.000	10000	1000	2	k-means++	30	0.008	0.0	0.005	30	0.003	0.001	0.000	2.835	0.624	See
2	KMeans_short	predict	0.002	0.000	10000	1	2	k-means++	30	0.009	0.0	0.005	30	0.003	0.000	0.000	9.414	4.436	See
3	KMeans_short	fit	0.134	0.005	10000	10000	2	random	30	0.036	NaN	0.005	30	0.003	0.068	0.002	1.967	0.103	See
4	KMeans_short	predict	0.002	0.000	10000	1000	2	random	30	0.008	0.0	0.005	30	0.003	0.001	0.000	2.980	0.678	See
5	KMeans_short	predict	0.002	0.000	10000	1	2	random	30	0.011	0.0	0.005	30	0.003	0.000	0.000	8.054	3.852	See
6	KMeans_short	fit	1.042	0.040	10000	10000	100	k-means++	16	0.123	NaN	0.005	23	0.003	0.578	0.032	1.802	0.121	See
7	KMeans_short	predict	0.003	0.000	10000	1000	100	k-means++	16	0.241	0.0	0.005	23	0.003	0.001	0.000	2.248	0.466	See
8	KMeans_short	predict	0.002	0.000	10000	1	100	k-means++	16	0.524	0.0	0.005	23	0.003	0.000	0.000	7.464	3.320	See
9	KMeans_short	fit	0.370	0.051	10000	10000	100	random	24	0.519	NaN	0.005	23	0.003	0.318	0.036	1.166	0.209	See
10	KMeans_short	predict	0.003	0.000	10000	1000	100	random	24	0.269	0.0	0.005	23	0.003	0.001	0.000	2.003	0.385	See
11	KMeans_short	predict	0.002	0.000	10000	1	100	random	24	0.477	0.0	0.005	23	0.003	0.000	0.000	7.656	3.566	See

LogisticRegression vs daal4py

All estimators share the following hyperparameters:

	value
penalty	l2
dual	False
tol	0.0001
C	1.0
fit_intercept	True
intercept_scaling	1
class_weight	NaN
random_state	NaN
solver	lbfgs
max_iter	100
multi_class	auto
verbose	0
warm_start	False
n_jobs	NaN
l1_ratio	NaN

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	class_weight	l1_ratio	n_jobs	random_state	n_iter	throughput	latency	accuracy_score	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	LogisticRegression	fit	15.053	0.084	1000000	1000000	100	NaN	NaN	NaN	NaN	[20]	[-0.07838136]	NaN	0.27	15.068	0.058	0.999	0.007	See
1	LogisticRegression	predict	0.000	0.000	1000000	1000	100	NaN	NaN	NaN	NaN	[20]	2.383341397078287	0.0	0.27	0.000	0.000	0.705	0.283	See
2	LogisticRegression	predict	0.000	0.000	1000000	1	100	NaN	NaN	NaN	NaN	[20]	7.905607056147397	0.0	0.27	0.000	0.000	0.376	0.262	See
3	LogisticRegression	fit	1.145	0.019	1000	1000	10000	NaN	NaN	NaN	NaN	[26]	[1.81630515]	NaN	0.27	1.088	0.017	1.053	0.024	See
4	LogisticRegression	predict	0.002	0.000	1000	100	10000	NaN	NaN	NaN	NaN	[26]	3.4045849118490943	0.0	0.27	0.004	0.000	0.614	0.097	See
5	LogisticRegression	predict	0.000	0.000	1000	1	10000	NaN	NaN	NaN	NaN	[26]	58.77801964677965	0.0	0.27	0.001	0.000	0.166	0.131	See

Ridge vs daal4py

All estimators share the following hyperparameters:

	value
alpha	1.0
fit_intercept	True
normalize	deprecated
copy_X	True
max_iter	NaN
tol	0.001
solver	auto
random_state	NaN

	estimator	function	mean_sklearn	stdev_sklearn	n_samples_train	n_samples	n_features	max_iter	random_state	throughput	latency	r2_score	mean_daal4py	stdev_daal4py	speedup	stdev_speedup	profiling
0	Ridge	fit	0.277	0.005	1000	1000	10000	NaN	NaN	0.288	NaN	1.0	0.285	0.003	0.973	0.019	See
1	Ridge	predict	0.012	0.000	1000	1000	10000	NaN	NaN	6.699	0.0	1.0	0.021	0.001	0.581	0.027	See
2	Ridge	predict	0.000	0.000	1000	1	10000	NaN	NaN	715.025	0.0	1.0	0.000	0.000	0.616	0.487	See
3	Ridge	fit	1.244	0.033	1000000	1000000	100	NaN	NaN	0.643	NaN	1.0	0.339	0.010	3.666	0.149	See
4	Ridge	predict	0.000	0.000	1000000	1000	100	NaN	NaN	4.823	0.0	1.0	0.000	0.000	0.657	0.374	See
5	Ridge	predict	0.000	0.000	1000000	1	100	NaN	NaN	9.002	0.0	1.0	0.000	0.000	0.659	0.528	See

Environment information

{
  "system_info": {
    "python": "3.8.8 | packaged by conda-forge | (default, Feb 20 2021, 16:22:27)  [GCC 9.3.0]",
    "executable": "/usr/share/miniconda/envs/sklbench/bin/python",
    "machine": "Linux-5.4.0-1046-azure-x86_64-with-glibc2.10"
  },
  "dependencies_info": {
    "pip": "21.1.1",
    "setuptools": "49.6.0.post20210108",
    "sklearn": "1.0.dev0",
    "numpy": "1.20.2",
    "scipy": "1.6.3",
    "Cython": null,
    "pandas": "1.2.4",
    "matplotlib": null,
    "joblib": "1.0.1",
    "threadpoolctl": "2.1.0"
  },
  "threadpool_info": [
    {
      "filepath": "/usr/share/miniconda/envs/sklbench/lib/libopenblasp-r0.3.15.so",
      "prefix": "libopenblas",
      "user_api": "blas",
      "internal_api": "openblas",
      "version": "0.3.15",
      "num_threads": 2,
      "threading_layer": "pthreads"
    },
    {
      "filepath": "/usr/share/miniconda/envs/sklbench/lib/python3.8/site-packages/scikit_learn.libs/libgomp-f7e03b3e.so.1.0.0",
      "prefix": "libgomp",
      "user_api": "openmp",
      "internal_api": "openmp",
      "version": null,
      "num_threads": 2
    }
  ],
  "cpu_count": 2
}